History Made Every Day

With the explosion in the number of images and videos on the Internet and ongoing amassing of huge volumes of visual data from digital cameras and mobile devices, we are unknowingly logging our own history. Consider that the number of digital cameras and camera phones in the world has surpassed one billion and that 500 billion consumer photos are taken each year. Given the world’s population of 6.4 billion, we can estimate that each person takes, on average, 78 photos per year, which is remarkable. However, the implication for life in cities is even more amazing. Consider that 3 percent of the Earth’s land is in cities and that half the world’s population currently lives in cities—a number that is expected to grow to 60 percent in 2030. If a typical photo covers 18 square meters of spatial area of land, and ignoring that photos are generally taken indoors and outdoors, then, on average, for cities across the entire planet, at least one consumer photo is taken for each 18 square meters of land per year. Thus, consumer photos alone cover the equivalent of the total surface area of all cities on the planet at least once per year. This becomes truly astounding if we consider bigger cities, like New York City, London, Paris, San Francisco, and Tokyo, where the density of consumer photos is many times higher (see Table 1). The propensity and ability to take photos in these cities is higher than average because people there are more likely to have digital cameras, these cities receive many tourists, and there are lots of attractions. However, ignoring these factors and basing the analysis only on population and land area, we can estimate that London gets 34 photos, Paris 29, and Tokyo 19 photos per unit area of land per year. The entire New York City across all five boroughs gets 14 photos per unit area per year. San Francisco across both city and county tops them all with 38 photos per unit area per year. Hence, for major cities in particular, people are doing a great job at passively capturing human existence. Of course consumer photos are not evenly spatially and temporally distributed, so we can’t conclude that every square inch of every city is captured many times over every year. But, the previous estimates don’t include traffic cameras, video surveillance systems, news cameras, street views, and overhead imagery (aerial, satellite, remote sensing) and so on. However,because thespatialandtemporalcoverage is not uniform in cities, we can also expect that entitiesofmost interesthavegreater representation, which means they will likely get hundreds and thousands of photos per year. Thus, automaticallycontextualizing thesedigitalphotos in terms of who, where, when, and what falls into the area of large-scale image matching. Chances are that there are many other photos for each person, place, object, and event of interest in cities. One challenge is that not all of the photos are publicly accessible, let alone in one place. For example, Flickr, which is one of the more popular photo-sharing sites, has 153 million geotagged photos. For a city like London, Flickr has 3.3 million geotagged photos. The other cities are summarized in Table 1. As a reference set for context for London, 3.3 million photos is too small. Assuming all of the geotagged photos were taken in the last year, which is not true, it would take Flickr 10 years to amass enough geotagged photos in London to equal its total land area. But, if we consider the who dimension of context, 3.3 million photos for a city of 7.8 million people is significant. If each photo in London, on average, captures one resident, Flickr should reach the same number of geotagged photos in London as the number of residents in a little more than two years. Assuming that the distribution of photos is not flat across people, many people in London are already captured many times over each year in the geotagged photos on Flickr. Beyond who, where, and when, the dimension of what is critical for recording history. Typically, what means what event or what activity. But, unlike people, places, and times, there [3B2-9] mmu2011030002.3d 1/8/011 16:9 Page 2